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Abstract. We study a stochastic recursive optimal control problem in which the cost functional is 
described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Some 
of the economic and financial optimization problems with volatility ambiguity can be formulated as such 
problems. Different from the classical variational approach, we establish the maximum principle by the 
linearization and weak convergence methods. 
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1 Introduction 


In economic theory, for a given consumption process (cs)o<s<t under probability P, Duffie and Epstein 
introduced the stochastic differential recursive utility 



f{y^{s),c{s))ds \Tt], 0 <t< T, 


( 1 . 1 ) 


and many optimization problems for the stochastic differential recursive utilities are well studied by Duffie 


and Skiadas [6| etc. In fact, the stochastic differential recursive utility is associated with the solution of a 
particular backward stochastic differential equation (BSDE). It is well known that the general BSDE was 


introduced by Pardoux and Peng [ 23 . Peng [2^ first generalized the classical stochastic optimal control 
problem to a new one in which the objective functional is defined by the solution of the following BSDE 
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US at time 0; 


-dy{t) = fit, xif), yit),z{t),uif))dt - zit)dB{t), 

y(T)= 0(x(r)), 


( 1 . 2 ) 


where B is a standard Brownian motion defined on a probability space (r2,B, P). From the BSDE point of 
view, El Karoui et. al. Q considered a more general class of recursive utilities defined as the solution of 
BSDEs. Thus, this new kind of stochastic optimal control problem is called the stochastic recursive optimal 
control problem. 

Chen and Epstein studied the stochastic differential recursive utility with drift ambiguity. The drift 
ambiguity in their context is described by a class of equivalent probability measures V. The stochastic 
differential recursive utility with drift ambiguity is defined as the lower envelope 


yit) = ess inf 2 /'^ (t), 0 <t <T, 


(1.3) 


where y^{t) is the solution of (11.11) at time t. They proved that y{t) of (jl.3l) can be characterized by a 
special BSDE and the corresponding recursive utility optimization problems with drift ambiguity still fall in 
the framework of the stochastic recursive optimal control problem. 

Many economic and financial problems involve volatility ambiguity (for the motivation to consider volatil¬ 
ity uncertainty, refer to Epstein and Ji [l^, [l^l. It is well known that volatility ambiguity is chracterized 
by a family of nondominated probability measures. In this case, dOJ can not be formulated as a classical 
BSDE, because it can not be modeled within a probability space framework. So we need a new framework 
to accommodate stochastic differential recursive utility with volatility ambiguity. 


fl 


Inspired by studying financial problems with volatility ambiguity (see [l|, |20|), Peng introduced a fully 
nonlinear expectation, called G-expectation E[-] (see 28| and the references therein) which does not require 
a probability space framework. Under this G-expectation framework (G-framework for short) a new type of 
Brownian motion called G-Brownian motion was constructed. The stochastic calculus with respect to the 
G-Brownian motion has been established. 




Recently, Hu et. al developed the BSDE theory under this G-expectation framework in 15. iia (see Soner 
et al. H for another formulation of fully nonlinear BSDE, called 2BSDE). In more details, they proved 
that the following BSDE driven by G-Brownian motion (G-BSDE for short) 

y W = ^ + /(s^ y(s)> zis))ds + /f gis, yis), z(s))fi(B)(s) 

-J^zis)dBis)-iKiT)-Kit)) 

has a unique triple of solution (y, z, K). In fact, in the volatility ambiguity case, (11.31) can be formulated 


as a special G-BSDE (see lOdlUj)- So the stochastic recursive utility optimization problem with volatility 


ambiguity is a special case of the following problem dll. The state equations are the following forward and 
backward SDEs driven by G-Brownian motion: for t G [0,T], 

dxit) = b{t,xit),u{t))dt + {t,x{t),uit))d{B^, B^){t) + a^{t,x{t),uit))dB^it), 

a;(0) = xq € R”, 
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-dy{t) = fit, xit), y{t),z{t),uit))dt + {t, x(t), z(t), uit))d{B\ B^)it) - zit)dB{t) - dK{t), 


y(T)= ^{xiT)). 

(1.4) 

The cost functional is introduced by the solution of the above BSDE at time 0, i.e., J(m(-)) = y(0). The 
stochastic recursive optimal control problem is to minimize the cost functional over the admissible controls. 
The stochastic maximum principle is an important approach to solve stochastic optimal control problems 


(see 1^ [l^ [is , 


ig 


21 


23, 


29 


31 


32 




3J, l36l|l. A local form of the stochastic maximum principle for the 


classical stochastic recursive optimal control problem was first established in Peng 2^. In this paper, we 
study the stochastic maximum principle for the problem dm) when the control domain is convex. 

Note that the solution y of (na at time 0 can be written as 


yo = K[(t)ix{T)) + / f{t,x{t),yit),zit),u{t))dt+ / {t,x{t),y{t), zit),u{t))d{B\ B^){t)] 

Jo Jo 

pT pT 


/ fit,xit),yit),zit),uit))dt + / g'-^t,xit),yit),zit),uit))d{B\B^)it)], (1.5) 
/o ^0 


= sup Ep[(j){xiT)) 
p&v 

where 7^ is a family of weakly compact nondominated probability measures (see [^). Thus, our stochastic 
recursive optimal control problem is essentially a ” inf sup problem”. Such problem is known as the robust 
optimal control problem, i.e., we consider the worst scenario by maximizing over a set of probability measures 
and then we minimize the cost functional. 

For the case / does not depend on [y, z) and = 0, i.e., 


J(u(-)) = E[(^(a;(r)) + / fit,x{t),u{t))di\, 


( 1 . 6 ) 


Xu 


33l| studied this problem. Based on the subadditivity of ]E[-], he obtained the variational inequality by 


the classical variational method. But he did not get the stochastic maximum principle since the sublinear 
operator E in his main theorem can not be deleted. It is worth to pointing out that the classical variational 
method can not be applied to obtain the variational inequality for our problem m. 

In the literatures, in order to derive the maximum principle for the classical stochastic recursive optimal 
control problem, one need to obtain the variational equation for the BSDE Cll). But in our context, since 
the K term of the solution of (d is a decreasing C?-martingale, it is unable to obtain the ’’derivative” 
for K in general. So we can not obtain the variational equation for the G-BSDE d- To overcome this 
difficulty, we introduce the linearization and weak convergence methods to directly obtain the derivative 
for the value function. By Minimax Theorem, the variational inequality on a reference probability P* is 
obtained. Based on the obtained variational inequality, we derive the stochastic maximum principle holds 
P*-a.s.. Furthermore, we prove that the obtained stochastic maximum principle is also a sufficient condition 
under some convex assumptions. 

The paper is organized as follows. In Section 2, we present some fundamental results on G-expectation 
theory. We formulate our stochastic recursive optimal control problem in Section 3. We derive the maximum 
principle in Section 4 and give the general results in Section 5. In Section 6, applying the obtained maximum 
principle, we solve a LQ problem. 
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2 Preliminaries 


16 


25-2811 for more 


We review some basic notions and results of (^-expectations. The readers may refer to 
details. 

Let n = Co([0, oo); be the space of valued continuous functions on [0,oo) with wq = 0 and let 
{B{t))t>o be the canonical process. For each fixed T > 0, set 

L,p(L!t) := MBih), • • • , B{tn)) : n > 1, fi, • • • , e [0, T], G 


where Cb.Lip{^‘^^‘^) denotes the space of bounded Lipschitz functions on Obviously, Lip{V,T) C 

Lip{^T') for T <T'. We also set 

OO 

Lipiyi') = 

n—1 

For each given monotonic and sublinear function G'(*) : ^ where denotes the collection of d x d 

symmetric matrices, there exists a bounded and closed subset F such that 

G(A) = isnptr[77^A], (2.1) 

^ 7er 

where denotes the collection oi dx d matrices. In this paper we only consider non-degenerate G, i.e., 

there exists some > 0 such that G{A) — G{B) > a^tT[A — B] for any A> B. Now, we define a functional 
E : Lip{Q) —>• R by two steps. 

Step 1. For X = (p{B{t -|- s) — B{s)) with t, s >0 and ip G G6.Lip(R'^), we define 

E[X] = M(t,0), 


where u is the solution of the following G-heat equation: 

dtu — G{D^^u) = 0, u(0, x) = <p{x). 

Step 2. For X = ip{B{ti) — B{to),B{t2) — B{ti), ■ ■ ■ ,B{tn) — B{t„-i)) with 0 < to < < tn and 

P S Gb.Lip(R‘^^”), we define 

E[X] = pn, 

where pn is obtained via the following procedure: 


Pi{xi,-- 

■ ,Xn-i) = E[(^(a;i,-- 


‘P2{X1,-- 

■ ,x„-2) = E[(,si(a:i,- 

j ^n— 2 ; 1 ) ^(^n— 2 


Pn = E[pn-l{B{ti) - B{to))]. 

The corresponding conditional expectation Ej of X with t = ti is defined by 

Et, [p{B{ti) - B{to), B{t2) - B{ti), • • • , B{tn) - S(t„_i))] 

= pn-i{B{ti) - B{to), ■■■ , B{ti) - S(ti_i)). 

It is easy to check that (Et)(>o satisfies the following properties: for each X,Y & Lip{Q), 
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(i) Monotonicity: li X >Y, then Et[^] > 

(ii) Constant preservation: Et[^] = X for X G Lipiyit)] 

(iii) Sub-additivity: Et[X -|- X] < Et[X] -|- Et[F]; 

(iv) Positive homogeneity: Et[XF] = X+Et[P] + X“Et[—P] for X G Lip{flt)] 

(v) Consistency: Es[Et[X]] = EsAt[X], specially, E[Et[X]] = E[X]. 

We denote by the completion of Lip^fl) under the norm HXUp^G = (E[|X|p])^/p for p > 1, 

similarly for Xq{Q,t). For each t > 0, Et[-] can be extended continuously to L\^{Vl) under the norm || • ||i,g- 
(n,LQ(f]),E) is called a G-expectation space. The corresponding canonical process {B{t))t>o is called a 
G-Brownian motion. 

Definition 2.1 A process {X{t))t>o called a G-martingale if X{t) G L]^(yit) and Es[X(t)] = X(s) for 

s <t. 

Remark 2.2 It is important to note that (—X(t))t>o may he not a G-martingale. 

Set 

V = {P : P is a probability on (12,23(12)), Ep[X] < E[X] for X G ^^(12)}. (2.2) 

Theorem 2.3 Let V he defined as in Then V is convex, weakly compaet and 

E[^] = maxi3p[^] for all ^ G ^^(12). 

V is called a set that represents E. 

The following proposition is important in our paper. 

Proposition 2.4 (^J) Let {Pn : n > 1} C P converge weakly to P. Then for each ^ G Lq(LI), we have 
Ep[£,]. 

Definition 2.5 Let Mq( 0,T) he the collection of processes in the following form: for a given partition 
{to, ■ ■ •, tAf} = ttt of [0, T], 

AT-l 

vA) = ^ 

3=0 

where A G Fjp(12tJ, i = 0,1, 2, • • •, TV - 1. 

We denote by Mq{0,T) the completion of Mq{0,T) under the norm HpUMg = \ri{s)\Pds]A^P for 

p > 1. The Ito’s integral r]{s)dB{s) is well defined for p G Mq{0,T). 
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3 Stochastic optimal control problem 

We first give the definition of admissible controls. 

Definition 3.1 u{-) is said to be an admissible control on [0,T], if it satisfies the following conditions: 

(i) u{-) : [0,T] X n —^ 17 where U is a nonempty convex subset o/R™; 

(ii) u{-) e M^( 0 ,r;R™) with /3 > 2 . 

The set of admissible controls is denoted hy IJ[0,T]. 

In the rest of this paper, we use the Einstein summation convention. 

Let u(-) G U[Q,T]. Consider the following forward and backward SDEs driven by G-Brownian motion: 
for t G [o,r], 

( dxit) = b(t,x(t), u(t))dt + h^Ht, x(t),u{t))d{B’’, B^){t) + ex''it, x(t), u(t))dB’’(t), 

(3.1) 

a:( 0 ) = Xo G R", 

I -dyit) = f{t,x{t),y{t),z{t),u{t))dt +g^^{t,x{t),y{t),z{t),u{t))d{B\B^){t) - z{t)dB{t) - dK{t), 

I y{T) = ^{x{T)), 

(3.2) 

where 

b: [0,T] xR" X C/^R'"; 

: [0, T] X R" X ^ R"; 

CT = [o-i,..., CT^] : [0, T] X R” X 17 R"^^; 

/ : [0,T] X R" X R X x 17 ^ R; 
gb : [0,r] X R”" X R X x 17 ^ R; 

: R" ^ R. 

Denote 

5g(0, T) = {h{t, Bt,M, ■ • •, Bt„M) :h,...,tnG[0,T],hG Gf,.L,p(R"+i)}; 

Sq{0,T) = {the completion of 5^(0, T) under the norm ||ry ||52 = {]E[sup(g[Q yj l^tP]}^}- 

For given u(-) G 71[0, T], x(-) and (y(-), z(-), K(-)) are called solutions of the above forward and backward 
SDEs respectively if x(-) G Mg(0,T;R"); (y(-),z(-)) G Sq{0,T) x Mg(0, T; K{-) is a decreasing 

G-martingale with 77(0) = 0 and K{T) G LQi^lx)] (13.11) and (13.21) are satisfied respectively. 

We assume: 

(HI) 6 , h^^, a, f, , 4> are continuous and differentiable in (x, y, z, u); 
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(H2) The derivatives of b, h’'^, a, f, 5 ®-^ , ij) in {x, y, z, u) are bounded; 

(H3) There exists a modulus of continuity oj : [0,oo) ^ [0,oo) such that for any t G [0,T], a;, x' G R", y, 
y' G R, z, z' G u, v! G R™, 

\(p{t, X, y, z, u) - (p{t, x',y', z', u')l < w(|x - x'\ + \y - y'\ + \z - z'\ + \u - u'\), 

where (p is the derivatives of 5, , a, /, ^ (x, y, z, u). 

We have the following theorems. 

Theorem 3.2 ff^l) Let assumptions (H1)-(H2) hold. Then SS. 1\) has a unique solution x{-). 

Theorem 3.3 (Un Let assumptions (H1)-(H2) hold. Then SS.2\) has a unique solution (y{-), z{-), K{-)). 

The state equation of our stochastic optimal control problem is governed by the above forward and 
backward SDEs (EB and (13.21) . The cost functional is introduced by the solution of the BSDE (13.21) at time 
0 , i.e., 

= 2 /( 0 ). 

The stochastic optimal control problem is to minimize the cost functional over W[0,T]. 

Remark 3.4 We point out that W[0,T] contains all feedback controls (see Hu and Ji Qy; . In the last 
section, we show that the optimal control of the LQ problem is a special kind of feedback control. 


In summary, our stochastic control problem is 


{ Minimize J{u{-)) 

subject to u(-)gW[0,T]. 


4 Stochastic Maximum Principle 

In this section, to ease the presentation we only study the case where hij = 0, gij =0 and / does not include 
z term. We will present the results for the general case in Section 5. 


4.1 Variational equation 


Let u(-) be optimal and {x{-),y{-), z{-), K{-)) be the corresponding state processes of (13.11) and (13.21) . Take 
an arbitrary u{-) G U[0,T]. Since W[0,T] is convex, then, for each 0 < p < 1, u{-) + p{u{-) — u(-)) G l([0,T]. 
Let {xp{-),yp{-),Zp{-),Kp{-)) be the state processes of (13.11) and (13.21) associated with u{-) + p{u{-) — u{-)). 

To derive the first-order necessary condition in terms of small p, let i(-) be the solution of the following 
SDE: 


dx{t) = [bx{t)x{t) + bu{t){u{t) - u{t))]dt + [al.{t)x(t) + al,{t){u{t) - u{t))]dB^(t), 


(4.1) 


;r(0) = 0, 
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where b^{t) = b^{t, x{t),u{t)), bu{t) = 6 „(t, x(t), u(t)), crj.(i) = al.{t,x{t),u{t)), al{t) = al{t,x{t),u{t)). 
In this paper, we define 

bixiit), ••• ,bix^it) 


bx{t) = 


bnx-i ; bjix^ (t) 


The other derivatives are defined similarly. 

Equation (14.11) is called the variational equation for SDE (13.11) . By Theorem 1.2 in 2^, there exists a 
unique solution x{-) £ Mq( 0,T;R") to equation (14.1|) . 

Set 

= P~^[xpit) - ^(t)] - x(t). 


Proposition 4.1 Assume (H1)-(H3) hold. Then 

(i) there exists a positive constant C such that E[| Xp{t) p] < C for 0 < p < 1; 

(ii) lim sup ]E[| Xp{t) p] = 0. 

In the following, we always use the constant C for simplicity, where C can be change from line to line. 
For prove this proposition, we need the following lemma. 


Lemma 4.2 Suppose that rj belongs to Mq{ 0,T). Then for each e > 0, there exists a positive number S 
such that | ry | < £ for any A £ B{[0, T]) x Tt with E[/p^ lA{t, u})dt] < S. 

Proof. Since p £ Mq{ 0,T), we have 

lim E[f I ?7 I /{|^|>Ar}dt] = 0 
Jo 

by Proposition 18 in 0|. Then for any e > 0, there exists a Ng such that E[/q^ | p \ I{\r]\>No}dt] < |. Take 
6 = 2 ^. For any A £ i3([0, T]) x J^t with E[/(j^ u})dt] < 6, we have that 


®[/o^ I V I lAdt] = E[/J’ I p I iI{\r,\>No}nA + /{|r,|<Aro}nA)d^] 

< I V I I{\v\>No}nAdt] +E[/J’ I p I I{\r,\<No}nAdt\ 

— ^[Jo I S I d[\rj\>NQ}di\ + E[Jp .^0^{|r;|<Afo}nA'^^] 

< e. 

This completes the proof. ■ 

Proof of Proposition 14.IL (i) From dUD and 6ID, we have 
/ 

dxp(t) = p~^[bp(t) - b{t) - p{bx{t)x{t) + bu(t){u(t) - u(t)))]dt 

' +P~^Kit) - - P{<{t)x{t) + ai{t){u{t) - u{t)))]dB^{t), 

Xp{0) = 0. 
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where bp{t) = b{t, Xp{t),u{t)+p{u{t)—u{t))), b{t) = 6(i, x{t),u{t)), crp{t) = cr^(t, Xp{t),u{t)+p{u{t)—u{t)))a.iid 
cr^{t) = a'^(t,x{t),u{t)). Let 

= [ b^{t,x{t) + Xp{x{t) + Xp{t)),u{t) + Xp{u{t) - u{t)))dX, 

Jo 

= / al.{t,x{t) + Xp{x{t) + Xp{t)),u{t) + Xp{u{t) - u{t)))dX, 

Jo 

Cp{t) = [Apit) - b^{t)]x{t) + [ [by,{t, x{t) + Xp{x{t) + Xp{t)), u(t) + Xp{u{t) - u{t))) - bu{t)]{u{t) - u{t))dX, 

Jo 

D"p{t) = [Bp{t) - (J^{t)]x{t) + [ [al(t, x{t) + Xp{x{t) + Xp(t)), u(t) + Xp(u(t) - u(t))) - <(t)](u(t) - u(t))dX. 

Jo 

Thus, 

dxp(t) = [Ap(t)xp(t) + Cp(t)]dt + [B^p(t)xp(t) + (<)]dB*(<), 

Xp(0) = 0 . 

Using Ito’s formula to | Xp(t) p, we get 
E[ I £p(t) p] 

= E[f 2(£p(s),Ap(s)xp(s) + Cp(s)}d.s+ [ (B;(s)ip(s) + U;(s), 

Jo Jo 

<C(E[ t \xpis) p ds]+Ip), 

Jo 


where C is a constant and 


Ip=E[ {\Cpis)\^ + \Dlis)\^)ds]. 


Applying Gronwall’s inequality, we obtain that 

E[| ipit) n < Ce^^Ip < Ce^^Ip. 


(4.2) 


Note that Cp{t) and Dp{t) are bounded by C"(| x{t) \ + \ u{t) — u{t) |), where C is a constant which is 
independent with p. Thus, E[| Xp{t) p] is bounded by some constant C for 0 < p < 1. 

(ii) By (IT:^ . we only need to prove that Ip —>■ 0 as p —>• 0. We first prove 


limE[ / I C'p(s) p ds] = 0. 
Jo 


Define 


Ep{t) = bx{t, x{t) + Xp{x{t) + Xp{t)),u{t) + Xp{u{t) - u{t))) - bx{t), 
Fp(t) = bu{t, x{t) + Xp{x{t) + ip(t)), u{t) + Xp(u(t) - u(t))) - bu{t). 

For iV > 0, set 

51, N = {| x{t) +Xp{t) \< N}, 

52, n = {| u{t) - u{t) \< N}. 
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(4.3) 


We have 

I Cp{t) |2= I Ep{t)d\x{t) + /q Fp{t)d\{u{t) - u{t)) |2 

< 2(/o I Ep{t) |2 d\ I x{t) p +fg I Ep(t) |2 dX I u(t) - u(t) p) 

< 2(/o' I Ep(t) |2 + Isi^ + IsiJdX | x(t) p 

+ /o I I I') 

< 2cD(2Np) I i(i) p +C(Isi, + IsiJ I m P 

+2a)(27Vp) I M(t) - u(t) |2 +C'(/sf^^ + IsiJ I ^(t) - u(t) ^ . 

By Lemmall^ for each £ > 0, there exists a 5 > 0 such that for any A G B{[0, T])xEt with E[/p^ lA{t, uj)dt] < 

6, we have that 


E[ [ I x{t) f lAdt] 

Jo 


< £, 


E[ / I u{t) — uit) p lAdt] < £. 


Note that 


®[/ {\x(t)+xp{t)\‘^+ \u{t)-u(t)\‘^)dt], 


then we can choose an TV > 0 such that E[jJ"(/sc ^ + Jgc ^)dt] < S, which implies that 

/ (ds-+ Is-^){\ x{t) p + I u{t)-u{t) \^)dt] < Ce. 


p —>-0 


Thus by (14.31) . it is easy to obtain liniE[Jp | Cp{s) p ds] = 0. Similarly, we can prove that liniE[J( 
Dp{s) p ds] = 0. Thus we get lim sup E[| Xp{t) p] = 0. □ 

Now let 


Set 


and 


where 


/p W = Xp{t), yp{t), u{t) + p{u{t) - u(t))), fit) = fit, xit),yit), uit)), 

fxit) = fj;it,xit),yit),uit)), fyit) = fyit,xit),yit),uit)), /„(t) = /„(t,x(<),u(t)). 

p* = {P ^ p \ Ep[KiT)] = 0} 

0“ = (/>a:(x(T))i(r)TO(r) + [ [/a;(s)i(s) +/„(s)(m(s) - u(s))]m(s)ds, 


mit) = exp{ / fyis)ds}. 

Jo 


Theorem 4.3 Suppose (H1)-(H3) hold. Then, for any u{‘) € U[0,T], there exists a P'^ such that 

K^ 2 /p( 0 )-y( 0 ) 


^ = EpulQA = sup Ep\Q^]. 

p^O p pgp. 


(4.4) 
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Remark 4.4 If B is the classical Brownian motion, then is the solution of the variational equation 

for BSDE at time 0. 

In order to prove this theorem, we need the following lemma. 

Lemma 4.5 Assume (H1)-(H3) hold. Then we have 

(i) E[| Jim(T) |]=o(p), 

(ii) ®[| U{s)xp{s)m{s)ds |] = o(l), 

(iii) E[| J 2 {s)m{s)ds |] = o{p), 
where 

Ji = (j){xp{T)) - (j){x{T)) - (j),,{x{T))px{T), 

J 2 {s) = if pis) - /(s)) - [f,,is)ixpis) - i(s)) + /y(s)(j/p(s) - yis)) + /„(s)p(u(s) - ?2(s))]. 

Proof, (i) 

= fo + K^piT) - xiT))dXixp{T) - x(T)) - (j)a;ixiT))px{T) 

= /q 4>xix{T) + \p{xp{T) + x{T))dXpixp{T) + i(T)) - (j)a:ixiT))px{T) 

= P fo(^x(x(T) + Xp(xp(T) + x(T)) - (j),,ix{T)))dXxiT) 

+p Jq f>x(x(T) + Xp{XpiT) + x{T))dXxpiT). 

Using the similar analysis as in Proposition 14.11 we can prove that 

limE[| [ ((/) 2 ,(x(r) + XpixpiT) + x(T)) - (l),^ix{T)))dX || x{T)m{T) |] = 0. 

It is easy to see 

E[| [\,ixiT) + XpixpiT)+xiT))dXxpiT)miT) |] < ^(Ell ip(r) \^])Hn\ HT) P])i 
Jo 

Then, by Proposition l4.ll 

limE[| [ (j)a;ix{T) + Xp{XpiT) + xiT))dXxp{T)m{T) |] = 0. 

P^O Jq 

(ii) 

®[l /o^/a:(s)ip(s)m(s)ds I] <C /o^E[| Xpis) II m(s) |]ds 

< C'/o^(IE[| Xpis) \^])^ds 

< CTi sup E[| Xpis) p]) 5 . 

0<s<T 

By Proposition 14.11 E[| fj' fxis)xpis)mis)ds |] —>■ 0 as p —0. 
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(iii) Set 

fi{s) = fi{s, x{s) + Ap(ip(s) + x{s)), y{s) + X{yp{s) - y{s)), u{s) + Xp{u{s) - u{s))) 
for I = x,y, u. Then 

Ms) = foifxis) - fM))dXp{xp{s) + x{s)) + lo ifyis) - fy{s))dX{yp{s) - y{s)) 
+ Io{fu{s) - fu{s))dXp{u{s) -u{s)). 

We only prove that 

®[| / / {fy{s)-fy{s))dX{yp{s)-M))m{s)ds\]=o{p). 

Jo Jo 

The proofs of the other terms are similar. 

By Proposition 2.15 in [l^, we have 

I yp{t) -M) P 

<C(Et[| Mp{T))-<^{x{T)) |2] 

I /(s,Xp(s),y(s),u(s) +p(u(s) -u(s))) “ f(s) P ds]). 

Then, by ProDOsition l4.ll 

sup E[| yp(t) -y(t) p] 

0<i<T 

<C(EHxp(T)-x(T)n 

+ /o^(lE[| Xp{t)-x{t) P] +E[p2 I u{t)-u{t) |2])dt) 

< C(E[p 2 I Xp{t) + x(t) P] 

+ /o^I Xp{t) + x{t) P] + I u{t) - u{t) Mdt) 

< Cp"^. 

Let a € (0,1) be fixed. For each iV > 0, we have 

®[l lo loify(s) - fvis))h\yp{s)-y{s)\>Np}dX{yp{s) - y{s))m{s)ds\] 

< cnio h\ypis)-vis)\>Np} I Vpis) -y{s) \ m{s)ds] 

< I yp{s) - y(s) |i+“ TO(s)ds] 

< I yp{s) - y(s) p (is])^(E[/p^ \m{s)\M;ds])M^ 

< 
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n\Io Ioifyi^)-fymh\^A s)+x{s)\>N} + I{\u{s)—u(s)\>N})dX(jJp{s) j/(s))m(s)(is|] 

s)+s(s)|>Ar} +-^{|ii(s)—'u(s)|>Af}) I yp('S) yis) I m{s)ds\ 

^ l^E[/o^(|ip(s) + ^(s)l“ + l“(s) - w(s)|“) I yp{s) - y{s) \ m(s)ds] 

< l^(E[/o^(|Sp(s) +x(s)P + |m(s) -u(s)|2)ds])t(]E[| yp{s)-y{s) P])5(]E[| m{s) \^])^ 

< w^P 


and 


E[l/o"/o(/vW-/yWK{|5p( s)+x{s)\<N}n{\yp{s)—y(s)\<Np}n{\u(s)—u{s)\<N}dX{yp[s) ?/(s))m(s)ds|] 

< uj{3Np) I yp{s) - y{s) \ m{s)ds] 

< Cu}{3Np){E[J^ I yp{s) - y{s) p ds])5(IE[/(f \m{s)\^ds])^ 

< Cu}{3Np)p. 

Thus we get for each > 0, 


T pi 


m / ifyis) - fyis))dX{yp{s) - y{s))m{s)ds W 


C 


0 ^0 


< Cu{mp)p + —p, 

which easily implies that E[| jJ’ fQifyis) - fy{s))dX{'yp{s) - y{s))m{s)ds |] = o{p). 

The proof is complete. ■ 

Proof of Theorem 14.31 

Step 1. We first prove that lirn exists. 

Consider 

yp(t) - yit) = <i^{xp{T)) - (/)(x(T)) + if pis) - fis))ds - J^izpis) - zis))dBis) 

-iKpiT)-Kpit)) + iKiT)-Kit)). 

It yields that 

Kit) + ypit) - yit) = KiT) + (ji^ixiT))pxiT) + Ji - f'^izpis) - z(s))dB(s) - iKpiT) - Kpit)) 

+ I^[f=ois)ixpis) - xis)) + fyis)iypis) - yis)) + fuis)piuis) - uis)) + J 2 is)]ds. 


Applying Ito’s formula to m(t)(Ar(t) + ypit) — yit)), we can get 

ypiO) - m = niKiT) + MxiT))pxiT) + Ji)m(T) 

+ J^ifxis)ixpis) - xis)) - fyis)Kis) + fuis)piuis) - uis)) + J 2 is))mis)ds]. 


(4.5) 


Note that 


KiT)miT) = / fyis)Kis)mis)ds + / mis)dKis), 
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then (14.51) becomes 


2/p(0) - y(0) = ^[{(t>x{.x{T))px{T) + Ji)m{T) + m(s)dK(s) 

+ ft^(fx(s)(xp(s) - x(s)) + fu(s)p(u(s) - u{s)) + J 2 (s))m(s)ds]. 

By Lemma 14.51 


yp(0)-J/(0)= H(l^x{x(T))px{T)m{T) + m{s)dK{s) 

+ lo(fxis)pxis) + fuis)p{u{s) - u{s)))m{s)ds] + o{p). 


(4.6) 


Since u{-) is an optimal control, we have 


2/p(0) - 2/(0) ^ m{s)dK{s) 


+ 0“] +o(l) > 0. 


(4.7) 


P P 

Note that Jo rn(s)dK{s) ^jg^j-gg^ggg as p J, 0. It yields that decreases. Since E)-] is sublinear, 

- r m(s)dK(s) , ~ m{s)dK(s) ^ 

jg Jo-w-w ^ jgjJo-w-w _ E _ 0 « ^ _E _ 01 X 1 

P P 

Thus, the limit of exists as p —)• 0. 

> p e 

Step 2. Then, we prove that there exists a P“ S V such that Ep^[K{T)\ = 0. 

Since V is weakly compact and 'm{s)dK{s) + p0“ G Lq^Ht), there exists a € V which depends 
on p and u{-) such that 


E[ 


fo m(s)dK(s) ^ ^ r /o m{s)dK{s) 


+ 0 “]. 


Thus (|4.7I) becomes 


11,(0) - 5(0) ^ M>)dK{s) ^ ^ ^ ^ 


P P 

Obviously, there exist a P“ G P and a sequence ^ P“ weakly as pn —>■ 0. By 


(4.8) 


we get 


Eppn,v.[ / m(s)(iPr(s)] = ?/p„(0) - ?/(0) - p„Pppn,«[0“] - o(p„). 


Note that 


|Pp.„,.[0“] |<E[|0“ |] <(X), 


it yields that Ppp„,ix[/g^TO(s)(iiir(s)] 0 as n ^ oo. Since m{s)dK{s) belongs to L'q(Q.t), it is easy to 

see that ^ ^ 

Eppn,-p[ f m{s)dK{s)] ^ Epp[ f m{s)dK{s)]. 

Jo Jo 

Thus we deduce that Epp[K{T)] = 0. 

Step 3. At last, we prove that lim = Pp„[ 0 “] = sup Pp[0“]. 

P^o P PeP* 
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By (14.811 and m{s)dK{s) < 0, 


;/p(o)-y(o) 

p 


<Spp..[0“]+o(l). 


Then 


Ij^MO)—lim £;pp„,.[0“] = Sp«[0“]. 

p—>^0 p n—¥oo 


For any P € 7^*, by (|4.7jl . we have 


yp(0)~v(0} _ ]^|' /n fn{s)dK{s) ^ ^ o(l) 

> 0«] + o(i) 

= £;p[0“]+o(l). 


It yields that 

> £:p[0“], yp eV*. 

P^O p 

Note that P“ GP*, then, by (14.91) and (14.101) . we obtain 

= Spu[0“] = sup Ep[Q^]. 

p^O p pgp. 

This completes the proof. □ 


(4.9) 


(4.10) 


4.2 Variational inequality 

We obtain the following variational inequality. 

Theorem 4.6 Suppose (H1)-(H3) hold. Then there exists a P* G V* such that 


inf Ep, [0“1 > 0. 
ueu[o,T] 

Proof. By Theorem 14.31 we can get for any u(-) G U[0,T], 

lim ~ = s.p £p(e“|>0. 

p-s-o p Pev* 


Then 


inf sup iilp[0“] > 0. 

uGW[ 0,T] pgp* 


It is easy to check that V* is convex and weakly compact, and for A G [0,1], u, u' G U\f), T], 

qAu+(1-A)u' ^ _ ;^)0«'_ 


Thus, by Sion’s minimax theorem, we obtain 


inf sup illp[0“] = sup inf £'p[0“]. 

u^'U[Q,T] p^'p* P^V* uGU[0,T] 
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Then, for each e > 0, there exists a. GV* such that 


inf Bps [0“1 > -e. 

«GW[0,T] 

Since V* is weakly compact, there exist a P* gV* and a sequence P* weakly as —>■ 0. Note that 

for any ■u(-) € U[Q, T], 

Ppen [0“] > —Sn- 

Letting e„ ^ 0, it yields that for any u{-) G W[0,T], 

Ep, [ 0 “] > 0 . 


Thus, we have 


This completes the proof. 


inf Ep, [0“1 > 0. 
uew[o.T] 


4.3 Maximum principle 

Consider the following kind of BSDE under P*: 


-dp{t) = [(fx{t)Y + {bx{t)Yp{t) + fy{t)p{t)]dt + {al.{t)yqYt)d{B\BYit) - qY^dByt) - dN{t), 

p{T)= Yx{x{TW, 

(4.11) 

where (p(t))te[o.T] G M|,.(0,T;R") = {p : rj is R^-valued progressively measurable and Ep*[J^ \pt\^dt] < 
oo}, {q{t))te[o,T] S Mp. (0,r;R"’^'^), (iVi)tG[o,T] & 7Vdpf‘(0, T; R") := {N : all R”-valued square integrable 
martingale that is orthogonal to P}. 

Remark 4.7 Note that B is only a continuous martingale under P* and the martingale representation 
theorem may not hold. So it is necessary to introduce the third term N which is orthogonal to B. 

Following El Karoui and Huang Q and Buckdahn et. al. [^, there exists a unique {p{-), q{-), N{■)) G 
Mp. (0,T;R") X Mp. (0, T; R"^'^) x Mp;^(0,T;R"’) which solves the adjoint equation (14.111) . Applying Ito’s 
formula to {x{t),m{t)p{t)), we obtain 

EpY(j)x;{x{T))x{T)m(T) + ( fx{s)x{s)m{s)ds] 

do 

= Ep.[[ {m{t){p{t),bu{t){u{t) - u{t))) P m{t){qYt),al{t){u{t) - u{t)))YYt))dt], 
do 

where r{t) = ( 7 *^(t)), d{B\BY{t) = YYt)dt. We define the Hamiltonian id : R” x R x R”" x R” x x 
[0, T] —>• R as follows: 

H{x,y,u,p,q,t) = {p,b{t,x,u)) + {qYYit,x,u))YYt) + f{t,x,y,u). 
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Thus 


Ep. [0“] = Ep. [/J" m{t){{bu{t))^p{t) + ifuit))'^ + u(t) - u(t))dt] 

= [/o^ m{t){{Hu{x{t), y{t),u{t),p{t),q{t), t))'^,u{t) - u{t))dt] 

= ^P‘ [!q P^{i)Hu{x{t),y{t),u{t),p{t),q{t),t){u{t) - u{t))dt]. 

By Theorem 14.61 £’p*[0“] > 0 for each m(-) G W[0,T], then we can get 

Hu{x{t),y(t),u{t),p(t),q{t),t)(u — u{t)) > 0, \/u G U, a.e., P* — a.s.. (4-12) 

We summarize the above analysis to the following stochastic maximum principle. 

Theorem 4.8 Suppose (H1)-(H3) hold. Let u{-) be an optimal control and {x{-),y{-), z{-), K{-)) be the cor¬ 
responding trajectory. Then there exist aP* GV* and {p{-),q{-),N{-)) £ Mp. (0,T;U") x Mp, x 

A4p;^(0, T; R”), which is the solution of the adjoint equation such that the inequality Ij.lSfj holds. 

4.4 Sufficient condition 

In this subsection, we give the sufficient condition for optimality. 

Theorem 4.9 Suppose (H1)-(H3) hold. Let u(-) G U[0,T] and P* G P* satisfy that 

Hu{x{t), y{t), u{t),p{t), q{t), t)(u — u[t)) > 0, Vu £ U, a.e., P* — a.s., 

where (x{-),y{-),z{-),K{-)) is the state processes of i3.1\) and \3.fA) corresponding to u{-) and (j>{-), q{-), N{■)) 
is the solution of the adjoint equation under P*. We also assume that H is convex with respect to x, 

y, u and (j) is convex with respect to x. Then u{-) is an optimal control. 

Proof. For any u(-) G h{[0,T], let {x{-),y{-),z{-),K{-)) be the corresponding state processes of (13.11) and 
(|3.2I) . Define f{t) := x{t) — x{t) and ^{t) := y{t) — y{t). Then ^(•) and r]{-) satisfy the following equations 
under P*: 

I d^{t) = [bx{t)^{t) + a{t)]dt + [al{t)f{t) + /3*(t)]dB*(t), 

[ ?(0)= 0, 

where 

a{t) := -bj;{t)^{t) + b{t, x{t),u{t)) - b{t, xlt),u{t)), 

P^t) := -al,{t)^{t) + a\t, x{t), u{t)) - a\t, x{f), u{t)), 

and 

j -dy{t) = [fx{t)f{t) + fy{t)v{t) + a{t)]dt - z{t)dB{t) - dK(t), 

I y{T)= p,{x{TmT) + P{T), 
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where z{t) := z{t) — z{t), 


a{t) := -fx{t)^{t) - fy{t)vi.t) + fit, xit), yit),u{t)) - f{t, x{t),y{t), u{t)), 

m := -M^irmiT) + cfixiT)) - mT)). 


For simplicity, set 


Hit) := Hixit),yit),uit),pit),qit),t). 

The definitions of H^if), Hyit) and Huif) are similar. Applying Ito’s lemma to m(t)p(t)) — r]it)mit) 
under P*, we can derive 


Ep. mr), miT)piT)) - (C(0), m(0)p(0)) - 7;(T)m(T) + 7?(0)m(0)] 

= + {rnit)q^it),l3^it))Y^it)+rnit)ait))dt - mit)dKit)] 

= Ep4loi-Hxit)fit) - Hyif)yit) P Hixit),yif),uit),pif),qit),t) - Hif))mit)dt - mit)dKit)] 

^ Io^-Hxit)f.it) -Hyit)qit) -Huit)iuit) -uit)) + Hixit),yit),uit),pit),qit),t) - 

The last inequality is due to the assumption and —mit)dKit) > 0. Note that H is convex with respect to 
X, y, u. We have 

-Hxit)fit) - Hyit)yif) - Huit)iuit) - uit)) > Hit) - Hixit),yit),uit),pit),qit),mit),t). 

It yields that 

Ep, liar), miT)piT)) - (e(0), m(0)p(0)) - ry(T)m(T) + ry(0)m(0)] > 0, 

which leads to Ap. [— /3(T)to(T) + ??(0)] > 0. Since (j) is convex with respect to x, we have that (iiT) > 0. 
Thus, 77 ( 0 ) > 0, which implies that u(-) is an optimal control. This completes the proof. ■ 


5 The general case 

In this section, we consider the general state equations. 

5.1 / includes z term 

Now we study the case in which the generator / of ( 13 . 21 ) includes the term z and we use the notations in 
Section 4. For simplicity, we assume that / only contains the term z, the other terms can be analyzed 
similarly as in Section 4. Similar to the proof of Theorem 14.31 we can get 

Kit) + ypit) - yit) = KiT) p (j)xixiT))pxiT) + Ji p /J Ap(s)(zp(s) - z(s))^ds 
- - -zis))dBis) - iKpiT) - Kpit)), 

where Ji is the same as in Section 4 and Ap(s) := /z(z(s) + A( 2 :p(s) — z(s)))dA. Following [2, we 

construct an auxiliary extended G-expectation space (fl, if (il), with 0, = Go([0, 00 ), and 


18 


^ G ^2d- 


G{A) = - suptr 

2 7Gr 

Let B{t))t>o be the canonical process in the extended space. It is easy to check that {B^,B^){t) = Sijt. 

Consider the equation 

dmp{t) = Ap{t)mp[t)dB{t), TOp(O) = 1. 

Applying Ito’s formula to mp{t){K{t) + yp{t) — y{t)), we can get 


A 


77 J 

I 


yp{0) - m = + Mx{T))pxiT) + Ji)mp(T)]. 


(5.1) 


Note that 


then m becomes 


K{T)mp{T) = f Ap{s)K{s)mp{s)dB{s) + [ mp{s)dK{s), 


2/p(0)-y(0) = E'^[(0x(a;(r))ps(T) + Ji)TOp(r) +/p mp{s)dK{s)] 

= E^[{(j)a:{x{T))px(T)m{T) + mp{s)dK{s) + Jimp{T) + J2], 


where J2 = 4>x{x{T))px{T){mp{T) — m{T)) and 

dm{t) = fz{t)m(t)dB{t), m(0) = 1. 

Similar to the proof of Lemma H31 we can obtain E^[ | Jimp(T) |] = o{p). By Proposition 3.8 in [l^, we 
can get 

[ l'2p('S) - ^(s)pds] < Cp. 

Jo 

Then similar to the proof of Proposition 14. 11 we can easily obtain E‘^[|x(T)(TOp(r) — m(T))|] = o(l). Thus 
we get 

= ±G^ fo (j)^(^x{T))x{T)m{T)] + 0(1). 

We can choose a sequence pk iO such that G V converges weakly to P“ G 'P and 

,i„ i'p.(°)-5(0) , SpWjiSM, 

k —^00 


Pk 


p^O 


p 


Jo mp^{s)dK{s) 




Pk 


+ (px{x{T))x(T)m{T)] = Epk,: 


Jo mp^(s)dK(s) 


Pk 


■ (j):^{x{T))x(T)m{T)], 


where V represents E'^[-]. It is easy to check that Ppfe,„[/p mp^{s)dK{s)\ — 5> 0 as fc — >■ cx). Note that 

E^[| f {mp{s) — m{s))dK{s)\] ^ 0 as p ^ 0, 
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then we can get Epk,u Uo m{s)dK{s)] —>■ 0 as fc oo. Similar to the proof of Theorem 14.31 we can get 
P“ G = {P g p : Ep[K{T)] = 0} and 

sup Ep[(pxix{T))x(T)m{T)] < liminf ^ ^ < limsup ^ ^ Pp„ [02;(a;(T))a;(T)m(T)], 
pg-p. p^o P p^O P 

which implies 

lim yp^i ^—= Ep^[(j)^{x{T))x{T)m{T)]. 

p^O p 

Similar to the proof of Theorem 14.61 there exists a P* G P*such that 

inf Pp. [(iii 3 ;(a;(r))a;(T)m(T)] > 0. 

ueu[o,T] ^ ' 

Now we introduce the following adjoint equation under P*: 

-dp{t) = {[{b^it))^ + + fzAt)Q^’Ht)}dt 

+ ic^iit)Vd^d{t)d{B\B^){t) + fzj{t)[q‘^’"{t) - f^,{t)P{t)]d{B\B^){t) 
-q^’^{t)dB’'{t) - - f^,{t)p{t)]dB^{t) - dN{t), 

p{T) = {Ux{T))r. 

Set T = a{Bt : t > 0) and P* = P* |p. We first show that (p(-), g^(-), iV(-)) G Mp, (0,r;]R”) x 
X Ad^i^(0,r;R"). For this we consider the following BSDE under {n,E,P*)-. 

-dp{t) = + /zi(t)«(t))^]p(i) + f^,{t)q^{t)}dt 

' E{<7i{t)Yq^{t)d{B\B3){t)-q\t)dB^{t)-dN{t), (5-3) 

p{T) = {MHtW- 

By 0, Qi the above BSDE has a unique solution {p{-), q{-), N{-)) G Mp.(0,T;R") x Mp.(0,T;x 
Af pi^(0, T; R"). It is easy to check that 

{p{-),q\-),f{-),N{.)) = {p{-),q{-),p{-)M-),N{.)) (5.4) 

is the unique solution of the adjoint equation (15.21) . Applying Ito’s formula to {x{t), m{t)p{t)) under P* and 
relation (EH), we can get 

Ep,[MxiT))xiT)m{T)] 

= Ep, [/o^ {{m{t)p{t),bu{t){u{t) - u{t))) + {m{t)f^,{t)p{t),al{t){u{t) - uit))) 

+ {m{t)q^ {t))dt]. 

We define the Hamiltonian P : R" x R x R^^'^ x R™ x R" x R"^'^ x [0, T] —>■ R as follows: 

H{x,z,u,p,q,t) = {p,b{t,x,u)) + {f:,,{z)p,a\t,x,u)) + {q\a\t,x,u))j"^ (t) + f{z). 
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Then 


Ep, [(j)x{x{T))x{T)m{T)] = Ep, [ / m{t)Hu{x{t), z{t), u{t),p{t), q{t),t){u{t) - u{t))dt]. 


Thus 


Hu{x{t), z{t), u{t),p{t), q{t),t){u — u{t)) > 0, Vu G U, a.e., P* — a.s.. 

Note that all the terms in the above inequality are measurable with respect to then we get 

Hu{x{t), z{t), u{t),p{t), q{t),t)(u — u{t)) > 0, Vu G U, a.e., P* — a.s.. 

We summarize the above analysis to the following theorem. 


(5.5) 


Theorem 5.1 Suppose (H1)-(H3) hold and f only depends on the term z. Let u{-) be an optimal control 
and {x{-),y{-), z{-), K{-)) be the corresponding trajectory. Then there exist a P* GP* and {p{-),q{-),N{-)) G 
Mp. (0, T; R”) X Mp. (0, T; x Alpi^(0, T; R"), which is the solution of the adjoint equation \5.S[) . such 

that the inequality i5.5\) holds. 


5.2 The general maximum principle 

In this subsection, we study the general case, i.e. the state equations are governed by (13.11) and (|3.2I) . We 
only list the main results since the proofs are similar as in section 4 and subsection 5.1. 

For this case, we introduce the following variational equation: 


/ 

dx(t) = [bxipxft) + bu(t){u(t) - u(t))]dt + [hfi(t)x(t) + hfi{t){u{t) - u{t))]d{B'^, Bp{t) 

' +[4(t)i(t) + <(t)(u(t) - u{t))\dBpt), 

i(0) = 0. 

Similarly, for some P* G P*, the following adjoint equation has a unique solution {p{-), q{-), N{■)) G 
M|.. (0,T;R") X M|,.(0,r;R”^‘^) x T; R"). 

-dp{t) = {{fPt)Y + [{bpt)Y + hMKiPV + fyitMt) + hAt)qpt)}dt 

+{(.9lPt)r + +4(t)g'(t) 

< (5.6) 

{alit))^qpt)}d{B\Bp{t) - qpt)dBpt) - dN{t), 
piT) = {<fPx(TW. 

Define the Hamiltonian iJ : R" x R x R^^'^ x R™ x R™ x R” x R”^"^ x [0, T] —>• R as follows: 

H{x,y,z,u,v,p,q,t) = {p,b{t,x,u)) + {p,E^ {t,x,u))-f'-pt) + {qPapt,x,u))Ypt) + {p{f^^{t,x,y,z,v) 

+9l\ {t, X, y, z, v)),a’- {t, x, (t) + f{t, x, y, z, u) + {t, x, y, z, {t), 

where i, j, I = 1,... ,d. 
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Theorem 5.2 Suppose (H1)-(H3) hold. Let u{-) be an optimal control and be the cor¬ 
responding trajectory. Then there exist aP* €V* and {p{-), q{-), N {■)) G Mp. (0, T; R") x M|,. (0, T; x 

T; R”), which is the solution of the adjoint equation 15.61) . such that 

Hu{x{t),y{t),z(t),u{t),u{t),p{t),q{t),t){u — u{t)) > 0, Vu G U, a.e., P* — a.s.. (5.7) 

In the following, we give the sufficient condition for optimality. 

Theorem 5.3 Suppose (H1)-(H3) hold. Let u(-) G U[0,T] and P* G V* satisfy that 

Hu{x{t),y{t),z(t),u{t),u(t),p(t),q{t),t){u-u(t)) >0, Wu € U, a.e., P* - a.s., 

where {x{-),y{-),z{-),K{-)) is the state processes of i3.1\) and i3.3\) corresponding to u{ ) and {p(-), q{-), N{■)) 
is the solution of the adjoint equation 15.61) under P*. We also assume that H is convex with respect to x, 
y, z, u and 4> is convex with respect to x. Then u(-) is an optimal control. 


6 LQ problem 


For simplicity, we suppose d = 1. In this case, 

G{a) = — ^a~), a G R, 

where ^ Consider the following LQ problem. The state equation is 


{ dx{f) = [A{t)x{t) + B{t)u{t) + b{t)\dt + [C{t)x{t) + D{t)u{t) + a{t)]dB{t), 

x{0) = Xq, Xq G R", 


( 6 . 1 ) 


where 6/[0, T] := {u(-) | ■u(-) G Mq(0,T; R"*)} and ^(O) C{-), B{-), D{-), b{-), a(-) are deterministic functions. 
The cost functional is 

'^(“(•)) = J [{Q{t)x{t),x{t)) -\-2{S{t)x{t),u{t)) + {R{t)u{t),u{t))]dt {Lx{T),x{T))], 

where Q{-), S{-), R{-) are deterministic functions. The stochastic optimal control problem is to minimize 
the cost functional over U[0,T]. 

In the following, the variable t will be suppressed. We suppose the functions satisfy the following condi¬ 
tions: 

/ 

G L“(0,r;R”''”), S G T°°(0,T;R’">'™), D GC{f},T-,W^^'^), 

' Q G L“(0, T; §„), S G L“(0, T; R™x"), R G ^(0, T; S,„), (6-2) 

6,ct G L2(o,r;R’^), L G S„ 

i? > 0, Q - SR-^S'^ >0, L > 0, (6.3) 
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where i? ^ 0 means that there exists a J > 0 such that R > 61 and similarly for L ^ 0. In this case, the 
Hamiltonian function is 


H{x, u,p^ q, t) = {p, Ax + Bu + 6) + (g, Cx + Du + (j)'j{t) + x) + 2{Sx, u) + {Ru, u)). 

Let u be an optimal control. By maximum principle which still holds for this case, there exists 
such that 

Ep,[K{T)]=0; 

B"’"p{t) + D'^q{t)j{t) + Sx{t) + Ru{t) = 0, under P*, 
where {p{-), q{-), N{-)) is the solution of the following adjoint equation under the probability P* 

—dp{t) = [Qx{t) + S'^u{t) + A^pit) + C'^ q{t)^{t)]dt — q{t)dB{t) — dN{t), 
p{T) = Lx{T). 

Suppose that 

p{t) = P{t)x{t) + ip{t) 

with P(-) G T], §„), (p(-) G C^([0,T],]R”). Applying Ito’s formula to p(t), we can get 


(6.4) 
a P* gP 

(6.5) 


( 6 . 6 ) 


(6.7) 


q(t) = P(t)C(t)x(t) + P{t)D{t)u{t) + P{t)a{t), 

Px + PAx + PBu + Pb + If + Qx + S'^u + A^p + C^qj = 0. 

Combining (16. 5p . (16.71) and the above two equalities, we can obtain that 

q=[PC- PD{R + D'^PDj)-\B^P + S + PC-f)]x -PD{R + PD-f)-\B'^p + Pa-/) + Per, (6.8) 

u = -{R + PD-/)-^[{B'^ P + S + PC-/)x + B^ p + Pa-f], (6.9) 

and the following Riccati equation for P 

/ 

P + PA + A^P + C'^PC-i + Q 

< -(P^P + S + D^PC-f)^{R + D'^PD-f)-^{B'^P + S' + D^PC-/) = 0, a.e. t G [0, T], (6A0) 

, P(T) = L, 

/ 

P+[A- B{R + D^PD-f)-^{B^P + S + D^PC-f)fp 
< +[C - D{R + D'^PD-f)-\B'^P + S + D^PC-/)fPa-f + Pb = 0, a.e. t G [0, P], (6-11) 

^ ^(T) = 0. 

It is important to note that P* is uniquely determined by the choose of 7. We choose -/{t) = It is well 
known that the Riccati equation (16.1011 has a unique solution P 0, and then equation (16.1111 has a unique 
solution p. In this case, the optimal control 

u = -{R + D'^PDa'^)-^[{B^P + S + PCa^)x + B'^ p + Paa% (6.12) 
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where 

{ dx(t) = [A(t)x(t) + B(t)u{t) + b{t)]dt + [C{t)x(t) + D(t)u{t) + a(t)]dB(t), 

(6.13) 

a;( 0 ) = Xo, Xo € K". 

In the following, we prove that the above u is the optimal control. 

Theorem 6.1 Suppose and i6.3\) hold. Then u defined in h6.12\) and W.1S\) is the optimal control, 
where P and (p are solutions for equations i6.10\} and \6.11]) with j(t) = . 

Proof. Let P* G V he the probability such that {B){t) = d^t. It is easy to check that p, q defined in (16.71) 
and (16.8p . and TV = 0 is the solution of the adjoint equation (16.61) under the probability P*. Also, it is easy 
to check that the Hamiltonian function H is convex with respect to x, u and 

Huix{t),u(t),p{t),q{t),t) =0, under P*. 

By Theorem 14.91 we only need to verify that Ep* [K{T)] = 0. Let I be the solution of the following ODE: 

/ 

V + ((/?, b) + ^d^ (Per, cr) 

< -\{B'^P + d^D'^Pa^iR + D'^PDd^)-^{B'^ip + d'^D'^Pa) = 0 , a.e. t G [ 0 ,T], 

1{T) = 0. 

Set 


Y{f) = \ {Px, x) + {p,x) + I, 

Z{t) = {Px + (p, Cx + Du + cr), 

K{t) — i Jq{P{Cx + Du + cr), Cx + Du + a)d{B){s) — J* G((P(Cx + Du + a), Cx + Du + a))ds. 

By applying Ito’s formula to Y and some simple calculations, we can get 

Y{t) = -{Lx{T),x{T)) + 2 ^ + 2{Sx,u) + {Ru,u)]ds — Z{s)dB{s) — {K{T) — K{t)), 

which implies that K{T) = K(T). Note that {P{Cx + Du + a),Cx + Du + a) >0, then we get 

K{T) = - f {P{Cx + Du + a),Cx + Du + a)d{{B){s) — d'^s). 

2 Jo 

Obviously, Ep* [K{T)] = 0. Thus u is the optimal control. ■ 

Remark 6.2 Using the same method, we can obtain the result for the state equation and cost functional 
containing the term (B). For the Riccati equation ib.KA) . we only need R -\- D^PDd^ > 0. This case will 
be discussed in our forthcoming paper. Note that R + D^PDgf < 0 may be hold, so the LQ problem may 
be infinite for some P G V, but it is finite under G-expectation. The reason of this is the uncertainty of 
probability measures, which is different from classical LQ problem. 
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In the following, we give an example to point out that the LQ problem with random coefficients is more 
difficult and P* is not the probability measure such that 

Example 6.3 We consider the following 1-dimensional state equation: 


x(t) = f as — {B){s)dB{s), 

Jo 

where a > a'^ is a constant. The cost functional is 

- {B)mu{t)\^d{B){t) + \x{T)\% 
By applying ltd’s formula to |a;(t)p, it is easy to check that 

J{u{-)) = - {B)m\u{t)\^ + l)d{B)it)]. 

Obvious, the optimal control w = 0 and P* G V satisfies 

rT pT 


E[ / (at - {B)it))d{B){t)] =Ep,[ {at - {B){t))d{B){t)]. 
Jo Jo 

By simple calculation, we can obtain P* is the probability measure such that 


where t* = a'^T{a + cT 
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2^-l 


{B)it) = [ {s.^I[o,t-]{s)+ a^I{t-,T]{s))ds, 
do 

It is easy to check that this u satisfies the maximum principle in Theorem 


7 Appendix 

The following proposition is about some further estimates for Theorems 14.31 and 14.61 which is interest of 
itself. 

Proposition 7.1 Suppose (H1)-(H3) hold. Then 

(1) for each u € U[0,T], there exists a P“ £ P* such that 


Epu[0“] 


lim Epu [ 

p-fO 


= sup Ep[0“], 

Pgp. 

- If (t), _ 

P i 


(2) there exists a P* € V* such that 


sup inf Ep[0“] = inf Ep.[0“], 

Pgp*ueM[0,T] u^U[0,T] 


inf (limEp* 
«G«[0.T] p^O 


P /o^ _ Q 
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Proof. (1) Consider 


2/p W - yit) = (t>{Xp{T)) - (j){x{T)) + J^ifpis) - f{s))ds - f^{z;{s) - z{s))dB{s) 

-{K;{T) - K;{t)) + {K{T) - K{t)). 

By Theorem 14.31 there exists a P“ S P* such that £'p«[0“] = suppgp. Pp[0“]. Note that K = 0 under 
probability P“. Similar as in the proof of Theorem 14.31 we can derive 


Ep^ 


^ /o Ms)dK;^{s) ^ ^ _ 2/"(0)-y(0) 


,( 1 ). 


By Theorem 14.31 

which implies that 


yU 0 )-m 


p^O 


lim Ep^ 

p—>-0 


= Pp^[0“], 


f-IoHs)dK^is) 


= 0 . 


(2) For any P € P*, similar analysis as in (1), we have 


m{s)dK^(s) 2/“(0)—2/(0) 

Ep[^ -= £;p[0“] _ ZPl^ -^ + o(i). 


P 


P 


Then, 


— L m(s)dK^(s) 

limEpl^ - — -= Pp«[0“] - Pp[0“] > 0. 

p-s-O p 


By Minimax Theorem, we can get 

inf sup Pp[0“] = sup inf Pp[0“]. 

u^U[Q,T] p^p* P^V* u^l^[0,T] 

By the proof of Theorem 14.61 we can obtain a P* S P* such that 


(7.1) 


Note that 


We deduce that 


sup inf Pp[0“]= inf Pp.[0“]. 
Pgp* «GW[o,r] ueuio,T] 


inf sup Pp[0“] = inf Pp40"], 
uGW[o,r]pgp. «gw[o,t] 

Pp„[0“]= sup Pp[0"] > Pp.[0"]. 

Pgp. 


inf (PpM0"]-Pp.[0“]) = 0. 

iieM[0,T] 


Taking P = P* in ([73]), it yields that 


inf (limPp. 
ueu[o,T] p->-o 


m(s)dP;(s) ^ 
P 


This completes the proof. ■ 
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